TIME SERIES ANALYSIS WITH INFORMATION THEORETIC LEARNING AND KERNEL METHODS By PUSKAL P. POKHAREL A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY

of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy TIME SERIES ANALYSIS WITH INFORMATION THEORETIC LEARNING AND KERNEL METHODS By Puskal P. Pokharel December 2007 Chair: Jose C. Principe Major: Electrical and Computer Engineering The major goal of our research is to develop simple and effective nonlinear versions of some basic time series tools for signal detection, optimal filtering, and on-line adaptive filtering. These extensions shall be based on concepts being developed in information theoretic learning (ITL) and kernel methods. In general all ITL algorithms can be interpreted from kernel methods because ITL is based on extracting higher order information (that beyond second order as given by the autocorrelation function) directly from the data samples by exploiting nonparametric density estimation using translation invariant kernel functions. ITL in general is still lacking in providing tools to better exploit time structures in the data because the assumption of independently distributed data samples is usually an essential requirement. Kernel methods provide an elegant means of obtaining nonlinear versions of linear algorithms expressed in terms of inner products by using the so called kernel trick and Mercer’s theorem. This has given rise to a variety of algorithms in the field of machine learning but most of them are computationally very expensive using a large Gram matrix of dimension same as the number of data points. Since these large matrices are usually ill-conditioned, they require an additional regularization step in methods like kernel regression. Our goal is to design basic signal analysis tools for time signals that extract higher order information from the data directly like ITL and also avoid the complexities of many kernel methods.